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Background / Context: 

Description of prior research and its intellectual context. 

The theoretical framework supporting the scaling up of this research-based, learning trajectories- 
focused mathematics intervention is an interconnected network of influences at school, 
classroom and student levels (Clements & Sarama, 2004), with faithful implementation at the 
center of the model. Sustained and faithful continuation of the intervention is theorized as a 
teacher outcome. Learning trajectories are at the core of the components of this intervention, 
guiding the curriculum, the instruction, the formative assessment, and the professional 
development for all these. Coherence of research and connection to practice have been theorized 
as supportive of sustainability (Boerst et al., 2010; O’Connell, 2012). Learning trajectory -based 
instruction provides coherence, and offers an emerging theory of teaching (Sztajn, Confrey, 
Wilson, & Edgington, 2012). This emerging theory of teaching provided guidance in the 
analysis and interpretation of this sustainability data. Moreover, the extant study from which 
these data are drawn has documented significant positive change for children within the 
treatment condition relative to children within the control condition. These results have been 
published recently across multiple journals (Clements, Sarama, Wolfe, & Spider, 2011). 

Purpose / Objective / Research Question / Focus of Study: 

Although there is growing interest in implementation fidelity, little research has been reported on 
the sustainability over time of faithful implementations of learning innovations (O’Connell, 
2012). This study examined the sustainability of teachers’ implementation fidelity to the 
components of a prekindergarten mathematics intervention two full years after external support 
ceased. Fidelity has been shown to decrease as time between interactions with developers 
increases (Datnow, 2005; Hargreaves, 2002). We hypothesized that an innovation based on 
coherent learning trajectories, wherein teachers increase their familiarity with mathematical 
goals, hypothesized developmental paths leading to the achievement of those goals, and activities 
designed to enable children to attain those goals, positively impacts teachers’ sustained and 
faithful implementation of the innovation. 

Setting: 

This research draws from a large-scale up study of professional development across two diverse 
research settings. Teachers were from 42 randomly assigned public schools in two cities (one 
distal), serving primarily low-income, ethnically diverse students 

Population / Participants / Subjects: 

Participants were 106 (72 treatment) prekindergarten teachers. Data from 64 masters-level 
teachers (1 male) in 26 schools across both sites are included here. Students enrolled at randomly 
assigned schools who returned parental consents were eligible. Student participants were four- 
year-olds (51% Female) of mixed ethnicity (53% African American, 21% Hispanic, 19% White, 
3.7% Asian Pacific, 1.8% Native American, and .6% Other). Most (82.33%) received free or 
reduced lunch, 13.5% had limited English proficiency, and 10% had an IEP. 
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Intervention / Program / Practice: 

Teachers participated in 13 professional development (PD) sessions over two years, and 
interacted with peer coaches and project staff mentors as they implemented the curriculum. The 
focus in PD sessions was not on just curricular instruction but on changing the way teachers 
think about mathematics instruction by focusing on learning trajectories (author reference; author 
reference; Simon, 1995). 

Research Design: 

Using longitudinal growth modeling, we examined fidelity to the curriculum at three time points 
across the course of the study: fall 2006, spring 2007, and spring 2009. The first two time points 
reflect observations of teacher fidelity during the data collection time period of the study. The 
spring 2009 time point reflects a return to these prekindergarten classrooms two years past their 
involvement with the project. 

Data Collection and Analysis: 

The main data source for this sustainability study was a classroom observation measure, rather 
than teacher self-reports, which make up the main body of evidence of sustainability (O’Connell, 
2012; Timperley, Wilson, Barrar & Fung, 2007). The Near Fidelity instrument was designed to 
evaluate whether or not, and to what degree, teachers were faithfully implementing the critical 
components of the curriculum, both on- and off-computer. The Near Fidelity instrument is 
comprised of multiple subsections. Observers rate agreement with curricula based statements on 
a 5 point scale (0 = neutral/not applicable). An example of the Likert scale questions within each 
subscale, with potential answers varying from “Strongly Disagree” to “Strongly Agree” is: “The 
teacher facilitated children’s responding: e.g., elicited many solution methods for one problem, 
encouraged elaboration of children’s responses, waited for and listened attentively to individual 
children, responded to errors as learning opportunities.” Project mentors collected data using this 
instrument, after participating in several training meetings and conducting interrater reliability 
visits over two years. Interrater reliability was 95%, arrived at through simultaneous classroom 
visits by rotating pairs of observers for 10% of all observations. Three time invariant covariates 
were selected for this analysis. The impact of site (dichotomous), years of experience teaching 
(range, 1-30+) and percentage of professional development attended. These three covariates 
seek to contextualize and account for both intercept and rate of growth differences across the 
three measured timepoints. 

Findings / Results: 

General Classroom. Within the General Classroom subscale, the intercepts are significantly 
different from one another across time points. Participant teachers continued to increase in the 
observed characteristics within the general classroom subscale. Neither sites nor percentage of 
PD attendance demonstrated significant impacts in initial levels. Teachers with more experience 
demonstrated lower intercept levels of GC across time compared to those of less experienced 
teachers. No significant differences in the rate of growth as a function of the included covariates 
were observed. This suggests a similar rate of growth across individual differences in site, years 
of experience, and percentage of PD attended (insert Table 1). 

Whole Group. Overall, the intercepts are significantly different from one another across time 
points. At each measurement timepoint, treatment teachers exhibited observed growth in their 
enactment and management of whole group settings within the instruction of mathematics while 
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maintaining fidelity to the core teaching methods within the professional development. 

Although the overall rate of growth was not significant, the proximal site exhibited significantly 
slower growth than the distal site. Site, however, was not a significant predictor of intercept 
differences across time. Teachers with more experience had higher levels of initial Whole Group 
fidelity than teachers with less experience. Teachers with greater percentage of attendance in PD 
had lower initial WG levels of fidelity across time. 

Small Group. Overall the intercepts are significantly different from one another across time 
points. Treatment teachers, again, demonstrate continued growth in their fidelity to the 
professional development program within the small group instructional setting in a pre-K 
mathematics lesson. Site has a significant impact on intercept differences such that teachers at 
the proximal site demonstrated significantly lower initial levels of Small Group (SG) across time 
relative to the distal site. Teachers with more experience demonstrated significantly lower initial 
levels of SG fidelity across time relative to teachers with less experience. Percentage of PD 
attended did not significantly impact initial levels of SG across time. The overall main effect for 
the slope was not significant, suggesting similar overall rates of growth across time points. 
Teachers with more experience demonstrated significantly higher rates of growth across time 
points relative to teachers with less experience. Site and percentage of PD attended were not 
significant predictors of growth in SG fidelity across time. 

Computer. Overall, the initial level of computer fidelity was significantly different across time 
and demonstrated significant growth between measurement periods. None of the time invariant 
predictors (Site, Yr. Exp, and percentage of PD attended) had a significant impact on initial 
levels of computer fidelity. Rate of change across time was negatively impacted by percentage 
PD attendance - such that greater attendance was related to a slower rate of growth for Fidelity 
to the Computer component. 

Conclusions: 

Teachers exposed to the PD during the intervention continue to demonstrate high levels 
of fidelity to the underlying curriculum two years past exposure and without continued project 
support (insert Figure 1). The percentage of PD attendance, however, did not act as a significant 
predictor for initial levels within any of the measured subscales. Importantly, however, greater 
PD attendance was positively related to change in initial levels across time. PD attendance did 
seem to negatively impact rate of growth within the whole group subscale. This could be due to 
the focus on small group activities and instruction within the overall PD program, where 
individual movement along learning trajectories was supported by the teacher. In accordance, a 
trend (p=.09) toward a positive impact of PD attendance on small group fidelity was also found. 

Site also demonstrated an inconsistent pattern across measured subscales. Initial levels 
are not significantly different except for fidelity to small group, for which Site 1 teachers are 
significantly lower. The rate of change is also not significantly different across subscales except 
for the WG subscale. 

Years of experience for the individual teacher demonstrated a varied pattern. Teachers 
with more experience had lower fidelity to the GC and SG components, consistent with prior 
research demonstrating that inexperienced teachers are more likely to sustain faithful 
implementation than experienced teachers (Gersten, Chard, & Baker, 2000; Stein et al., 2008), 
while experienced teachers may be more resistant to change (Coburn, 2004). Teachers with 
more experience demonstrated significantly faster rates of growth than their less experienced 
colleagues, and higher initial levels of fidelity to the whole group component. Overall, teachers 
with more experience may have gravitated toward and excelled at the whole group format as it 
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most closely mirrors their prior experience in PreK mathematics instruction. Exposure and time 
with the curriculum, however, seemed to indicate a rapid growth in experienced teachers’ fidelity 
to the SG component, perhaps indicating increasing understanding of the concepts and 
procedures of small group instruction. 

The learning trajectory based instruction engaged in by the prekindergarten teachers in 
this study may have provided a coherent program of teaching and learning, and may have 
promoted the statistically significant levels of fidelity found in this study. Teachers taught the 
curriculum with increasing fidelity as time went by, while no longer supported by research 
project staff. They seemed to have internalized the program (Timperley, Wilson, Barrar, & Fung, 
2007), and to have made sense of the curricular activities involved with whole group, small 
group, and other components, within an overall structure of learning trajectories that progressed 
toward a known mathematical goal. By engaging in the initial PD, and then, possibly becoming 
empowered by their own knowledge of the trajectories and the ways to support learners through 
the trajectories, they became progressively more faithful to the intended program, instead of 
drifting from it as time elapsed and support disappeared, as has been documented by other 
studies when external support is withdrawn (Datnow, 2005; Hargreaves, 2002). 

The coherence of a model of PD, curriculum, instruction, and assessment based on 
learning trajectories (see Daro, Mosher, & Corcoran, 2011) may provide the conditions for 
promoting sustainability in teacher practices, and may be particularly beneficial in addressing the 
climate of low expectations in urban schools (Johnson & Fargo, 2010), as teachers increase their 
understanding of the capacities of all children to learn mathematics. 

As teachers come to understand children’s probable developmental paths and become 
adept at anticipating children’s strategies and misconceptions, their teaching practices may 
become more grounded and solidified. As they monitor for students’ multiple models, and probe 
for the ways in which students mathematical thinking fits the structure of the trajectory, their 
teaching practices may become reinforced as student reactions provide positive feedback for 
their practices (Guskey, 2002). Teachers who demonstrate sustained fidelity of implementation 
to a program that has demonstrated improved child achievement could have an impact on many 
more students than teachers who implement with fidelity only during treatment. 
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Table 1. Descriptives and standardized estimates for Fixed and Random parameters 
for each best fitting modeled Fidelity subscale 



GC 

WG 

SG 

CP 

# of items 

5 

7 

25 

12 

Total 

Possible 

14 

28 

75 

40 

Reliability 

a=.60-.66 

ct=.81-.83 

a=.90-.95 

a=.73-.84 

Means 

Intercept 

5.49 (.760)** 

10.58 (1.48)** 

4.69 (1.38)** 

5.15 (1.34)** 

Slope 

5.38 (3.64) 

.154 (1.04) 

3.32 (2.32) 

2.64 (.657)** 

Random Effects 




Intercept 

Site 

-.060 (.156) 

.132 (.144) 

-.436 (.194)* 

-.234 (.204) 

Yr. Exp 

-.260 (.128)* 

-.234 (.092)* 

-.342 (.111)** 

.019 (.212) 

PD % 

.277 (.212) 

-.002 (.151) 

.344 (.203) 

.260 (.174) 

Slope 





Site 

-.746 (.550) 

-.383 (.153)* 

.148 (.394) 

-.017 (.143) 

Yr. Exp 

.607 (.510) 

.042 (.155) 

.830 (.159)** 

-.154(212) 

PD % 

.387 (.808) 

.112 (.226) 

-.418 (.351) 

-.241 (.122)* 

Variances 

Intercept 

.836 (.124)** 

.940 (.050)** 

.401 (.159)* 

.857 (.117)** 

Slope 

a 

.829 (.175)** 

a 

.922 (.076) 

Model Fit 

X 2 (df) 

15.80(6) 

9.64 (4) 

2.31 (6) 

7.23 (5) b 

RMSEA 

.152 (.062-245) 

.141 (.015-. 257) 

.000 (.000 -072) 

.085 (.000-. 211) 

SRMR 

.062 

.116 

.046 

.074 
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Figure 1 . Fidelity of implementation is displayed over three time points for each 
subscale. Subscales were measured on separate scales. Scale information is in Table 1 
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